chdkfandomcom-20200222-history
ExMem - enabling and loading CHDK into extended memory.
The core CHDK code now include some changes made for EXMEM (changeset 1124), including the code to build and load CHDK itself into the EXMEM memory space. This article describes setting up, testing and using EXMEM including loading CHDK itself into this memory. Also see http://chdk.setepontos.com/index.php?topic=5980.0 for additional information. It is based on work done on the G12 & SX30 ports. First here's a simplified memory map for recent dryos cameras like the G12 & SX30 (not to scale). The numbers are in HEX, the question marks are values that change from camera to camera: Without CHDK variables heap/stack buffers (e.g. RAW) exmem pool vid buffers |----|-----------|------------------|----------------------|---------------------|-----------| 0 1900 ? 4000000 ? ? MAXRAMADDR With CHDK (default memory) = variables CHDK heap/stack buffers (e.g. RAW) exmem pool vid buffers |----|-----------|----|-------------|----------------------|---------------------|-----------| 0 1900 ? ? 4000000 ? ? MAXRAMADDR With CHDK (exmem memory) = variables heap/stack buffers (e.g. RAW) exmem pool CHDK vid buffers |----|-----------|------------------|----------------------|-------------|-------|-----------| 0 1900 ? 4000000 ? ? ? MAXRAMADDR Moving CHDK into the EXMEM area means that the Canon heap is left at its original size, while still making more memory available for CHDK - everyone wins! In order to get this working on a camera there are a few steps involved: 1. Find MAXRAMADDR and add to PLATFORM/PLATFORMSUB/makefile.inc if not already there. 2. Find the exmem_alloc routine in the firmware, and enable EXMEM in the build (OPT_EXMEM_MALLOC). 3. Define a value for EXMEM_HEAP_SIZE. 4. Enable OPT_EXMEM_TESTING to determine if the allocated memory is safe to use. 5. If necessary, determine the value for EXMEM_HEAP_SKIP. 6. Turn off EXMEM testing and let CHDK use the EXMEM memory. 7. Enable CHDK to load into EXMEM. Step 1. MAXRAMADDR defines the upper limit of the cameras RAM. If it has not been set for your camera you need to determine what the value is and add it to the PLATFORM/PLATFORMSUB/makefile.inc file. For example cameras with 128MB of RAM will have MAXRAMADDR=0x07FFFFFF, cameras with 64MB of RAM will have MAXRAMADDR=0x03FFFFFF. Search in the firmware for a function that starts like the below (the addresses and values will be different): ROM:FF890270 EXPORT _sub_FF890270__ExMemMan.c__0 ROM:FF890270 _sub_FF890270__ExMemMan.c__0 ; CODE XREF: sub_FF88FD1C+AC ROM:FF890270 STMFD SP!, {R4-R8,LR} ROM:FF890274 LDR R6, =0x3BFC8 ROM:FF890278 MOV R4, R0 ROM:FF89027C MOV R0, #0 ROM:FF890280 MOV R5, #0x48000000 <<<<<<<<<< want this value ROM:FF890284 ROM:FF890284 loc_FF890284 ; CODE XREF: _sub_FF890270__ExMemMan.c__0+38 ROM:FF890284 LDR R1, R6,R0,LSL#3 We need the value marked in the code above and it will probably be either 0x48000000 (128MB) or 0x44000000 (64MB). The value will have the 'uncached' memory bit set (0x40000000), subtract that and the result is the amount of RAM on the camera. Set the MAXRAMADDR value in makefile.inc to be one less than this value. In the example above MAXRAMADDR will be 0x7ffffff (0x8000000 - 1). Step 2. This Is covered in reyalp's first post at the link above. Step 3. The default value for EXMEM_HEAP_SIZE in the code is 2MB if you don't define a value. You should add a 4MB value into PLATFORM/PLATFORMSUB/makefile.inc. Starting with 4MB is needed in case the camera has video buffers that need to be skipped (see later). PLATFORM/PLATFORMSUB/makefile.inc: EXMEM_BUFFER_SIZE=0x400000 # Amount of EXMEM memory to allocate for CHDK = 4MB Step 4. Enable OPT_EXMEM_TESTING in buildconf.inc. Rebuild CHDK and load it onto your camera. This enables exmem testing, which allocates the memory; but does not use it. instead it checks the memory for corruption. You should now have 'OK" displayed at the bottom left of the LCD. Test the camera & CHDK functions as much as possible. If the OK disappears and is replaced by a bunch of numbers then this means that the EXMEM memory has been corrupted by some camera function (most likely recording a video and using the video buffers shown in the memory map above). The info displayed on two lines: s:XXXXXXXX e:YYYYYYYY f:AAAAAAAA l:BBBBBBBB c:NN XXXXXXXX = start address of the EXMEM allocated memory buffer YYYYYYYY = end address of the EXMEM allocated memory buffer AAAAAAAA = first corrupt memory location BBBBBBBB = last corrupt memory location NN = number of corrupt memory locations found If the first corrupt location is at the start of the EXMEM allocated memory buffer then reduce the size of EXMEM_HEAP_SIZE to 2MB and try again. If that still fails then you probably won't be able to use EXMEM. If the first corrupt location is not at the start of the memory then it is most likely a problem with the video buffers, use step 5 below to try and fix this. If everything works and no corruption occurs you should be good to go to step 6. Step 5. In testing this on the G12 and SX30 it became apparent that the camera was using some of the EXMEM allocated memory. It was found that this memory was for buffers used during video recording and playback (possibly for H.264 encoding/decoding). The exmem_alloc function was not correctly locking this memory out of being allocated to CHDK. At this stage a proper fix for this has not been found and the following is a work around to the problem. These video buffers are approx 3MB in size. If you performed step 1 you will have already found the place where the video buffer address can be located, if not use the instructions in step 1 to find the function that tells us how much RAM the camera has. A bit later on in this function is the video buffer address: ROM:FF890270 EXPORT _sub_FF890270__ExMemMan.c__0 ROM:FF890270 _sub_FF890270__ExMemMan.c__0 ; CODE XREF: sub_FF88FD1C+AC ROM:FF890270 STMFD SP!, {R4-R8,LR} ROM:FF890274 LDR R6, =0x3BFC8 ROM:FF890278 MOV R4, R0 ROM:FF89027C MOV R0, #0 ROM:FF890280 MOV R5, #0x48000000 ROM:FF890284 ROM:FF890284 loc_FF890284 ; CODE XREF: _sub_FF890270__ExMemMan.c__0+38 ROM:FF890284 LDR R1, R6,R0,LSL#3 ... ROM:FF8902AC CMP R4, #0xB ROM:FF8902B0 CMPNE R4, #0xC ROM:FF8902B4 CMPNE R4, #0xD ROM:FF8902B8 BNE loc_FF8902C4 ROM:FF8902BC CMP R5, #0x48000000 ROM:FF8902C0 LDREQ R5, =0x47D08A00 <<<<<<<<< need this value for video buffer address ROM:FF8902C4 ROM:FF8902C4 loc_FF8902C4 ; CODE XREF: _sub_FF890270__ExMemMan.c__0+48 ROM:FF8902C4 LDR R7, =dword_FFBBB370 On the G12 & SX30 the line marked above tells us the video buffer address, again this has the uncached memory bit set so remove that (in this case the value we want is 0x07d08a00 after removing the uncached bit). Subtract this value from the memory size (0x08000000 in this case) to get the size of the video buffer memory (the camera uses everything from this address to the top of RAM). Add this size value as EXMEM_HEAP_SKIP to PLATFORM/PLATFORMSUB/makefile.inc. In this case we would add 'EXMEM_HEAP_SKIP=0x2F7600' since 0x08000000 - 0x07d08a00 = 0x2f7600. Rebuild and try the exmem testing again (step 4). Step 6. At this stage you should have a working exmem buffer that does not get corrupted and hopefully at least 2MB in size. Disable the OPT_EXMEM_TESTING option and rebuild. Load this onto the camera and you should now have CHDK running with a lot more free memory (Miscellaneous stuff --> Show memory info). The load address will still be in the low memory heap area. Thoroughly test the camera and CHDK functionality. If all is well go onto step 7, otherwise adjust the EXMEM_HEAP_SIZE value and try again. Step 7. The final step is to move CHDK itself up into EXMEM. To do this the value of MEMISOSTART in PLATFORM/PLATFORMSUB/makefile.inc needs to be changed. The formula for MEMISOSTART when loading CHDK into high (exmem) memory is: MEMISOSTART = MAXRAMADDR+1 - EXMEM_HEAP_SKIP - EXMEM_BUFFER_SIZE - 32 You have to calculate the actual value and insert it here, you can't use a formula (at least I haven't been able to). From my examples above the value would be: 0x7ffffff + 1 - 0x2f7600 - 0x400000 - 32 = 0x79089e0 (Now you know why the other values are in this file, so you can see them to do the calculation) The extra -32 at the end is because exmem_alloc allocates 64 bytes extra, with a 32 byte buffer before and after the block returned. This change to MEMISOSTART should be made conditional so that you can disable all this and easily revert back to the default non-exmem version as shown below (make sure to use your original MEMISOSTART value in the else part, not the value shown here): ifdef OPT_CHDK_IN_EXMEM MEMISOSTART=0x79089e0 # MAXRAMADDR+1 - EXMEM_HEAP_SKIP - EXMEM_BUFFER_SIZE - 32 (exmem allocates 64 bytes extra, 32 before and 32 after block allocated) else MEMISOSTART=0x16E820 # original non-exmem value endif A change is also required in the 'boot.c' code for the camera firmware version. This change is to set the built-in heap start address back to the default value when loading CHDK into exmem memory. Otherwise the CHDK startup code will try and put the Canon heap into exmem which will probably crash. Find the code in boot.c that sets the heap start address and change it as shown below (again make sure to use the correct address for your camera and firmware version). #if defined(OPT_CHDK_IN_EXMEM) "LDR R0, =0x16E820\n" // use original heap offset since CHDK is loaded in high memory #else "LDR R0, =new_sa\n" // otherwise use patched value "LDR R0, R0\n" // #endif This code above also shows the final piece - the OPT_CHDK_IN_EXMEM option needs to be enabled in buildconf.inc. Buildconf.inc should now look like this with OPT_EXMEM_ALLOC and OPT_CHDK_IN_EXMEM both enabled: # experimental - allocate memory from alternate heap. Default 2MB from EXMEM_RAMDISK # See exmem_malloc_init in generic/wrappers.c OPT_EXMEM_MALLOC=1 # Set OPT_CHDK_IN_EXMEM to build and load CHDK into EXMEM memory # Need to define MEMISOSTART and EXMEM values in PLATFORM/PLATFORMSUB/makefile.inc (see G12 or SX30 for example) # (if setting this must also set OPT_EXMEM_MALLOC !) OPT_CHDK_IN_EXMEM=1 # set this to enable exmem corruption testing (see wrappers.c). (Don't set OPT_CHDK_IN_EXMEM with this !) #OPT_EXMEM_TESTING=1 Rebuild and load onto your camera. Show memory info should now show a load address up in the high exmem area plus plenty of free memory. Category:Development Category:CHDK internals